You have 2 free member-only stories left this month.

Image for post
Image for post
Awesome Atom Functionalities that make it a worthy editor for Python

How to Create the most Awesome Development Setup for Data Science using Atom?

Github + Markdown + Stack Overflow + Autocomplete + Jupyter

Before I even begin this article, let me just say that I love iPython Notebooks, and Atom is not an alternative to Jupyter in any way. Notebooks provide me an interface where I have to think of Coding one code block at a time,” as I like to call it, and it helps me to think more clearly while helping me make my code more modular.

Yet, Jupyter is not suitable for some tasks in its present form. And the most prominent is when I have to work with .py files. And one will need to work with .py files whenever they want to push your code to production or change other people’s code. So, until now, I used sublime text to edit Python files, and I found it excellent. But recently, when I looked at the Atom editor, my loyalties seemed to shift when I saw the multiple out of the box options provided by it.

Now, the real power to Atom comes from the various packages you can install. In this post, I will talk about the packages that help make Atom just the most hackable and wholesome development environment ever.

Installing Atom and Some Starting Tweaks

Before we even begin, we need to install Atom. You can do it from the main website here. The installation process is pretty simple, whatever your platform is. For Linux, I just downloaded the .deb file and double-clicked it. Once you have installed Atom, You can look at doing some tweaks:

  • Open Core settings in Atom using Ctrl+Shift+P and typing settings therein. This Ctrl+Shift+P command is going to be one of the most important commands in Atom as it lets you navigate and run a lot of commands.
Image for post
Image for post
Accessing the Settings window using Ctrl+Shift+P
  • Now go to the Editor menu and Uncheck “Soft Tabs”. This is done so that TAB key registers as a TAB and not two spaces. If you want you can also activate “Soft Wrap” which wraps the text if the text exceeds the window width.
Image for post
Image for post
My preferred settings for soft-wrap and soft-tabs.

Now, as we have Atom installed, we can look at some of the most awesome packages it provides. And the most important of them is GitHub.

1. Commit to Github without leaving Editor

Are you fed up with leaving your text editor to use terminal every time you push a commit to Github? If your answer is yes, Atom solves this very problem by letting you push commits without you ever leaving the text editor window.

This is one of the main features that pushed me towards Atom from Sublime Text. I like how this functionality comes preloaded with Atom and it doesn’t take much time to set it up.

To start using it, click on the GitHub link in the right bottom of the Atom screen, and the Atom screen will prompt you to log in to your Github to provide access. It is a one-time setup, and once you log in and give the token generated to Atom, you will be able to push your commits from the Atom screen itself without navigating to the terminal window.

Image for post
Image for post
Image for post
Image for post
Click on GitHub and then log in

The process to push a commit is:

  • Change any file or multiple files.
  • Click on Git on the bottom right corner.
  • Stage the Changes
  • Write a commit message.
  • Click on Push in the bottom right corner.
  • And we are done:)

Below, I am pushing a very simple commit to Github, where I add a title to my Markdown file. Its a GIF file, so it might take some time to load.

Image for post
Image for post
Committing in Atom

2. Write Markdown with real-time preview

I am always torn between the medium editor vs. Markdown whenever I write blog posts for my site. For one, I prefer using Markdown when I have to use Math symbols for my post or have to use custom HTML. But, I also like the Medium editor as it is WYSIWYG(What You See Is What You Get). And with Atom, I have finally found the perfect markdown editor for me, which provides me with Markdown as well as WYSIWYG. And it has now become a default option for me to create any README.md files for GitHub.

Using Markdown in Atom is again a piece of cake and is activated by default. To see a live preview with Markdown in Atom:

  • Use Ctrl+Shift+M to open Markdown Preview Pane.
  • Whatever changes you do in the document will reflect near real-time in the preview window.
Image for post
Image for post
Markdown Split Screen editor

3. Minimap — A navigation map for Large code files

Till now, we haven’t installed any new package to Atom, so let’s install an elementary package as our first package. This package is called minimap, and it is something that I like to have from my Sublime Text days. It lets you have a side panel where you can click and reach any part of the code. Pretty useful for large files.

To install a package, you can go to settings and click on Install Packages. Ctrl_Shift+P > Settings > + Install > Minimap> Install

Image for post
Image for post
Installing Minimap or any package

Once you install the package, you can see the minimap on the side of your screen.

Image for post
Image for post
Sidebar to navigate large files with ease

4. Python Autocomplete with function definitions in Text Editor

An editor is never really complete until it provides you with some autocomplete options for your favorite language. Atom integrates well with Kite, which tries to integrate AI and autocomplete.

So, to enable autocomplete with Kite, we can use the package named autocomplete-python in Atom. The install steps remain the same as before. i.e. Ctrl+Shift+P > Settings > + Install > autocomplete-python> Install. You will also see the option of using Kite along with it. I usually end up using Kite instead of Jedi(Another autocomplete option). This is how it looks when you work on a Python document with Kite autocompletion.

Image for post
Image for post
Autocomplete with Kite lets you see function definitions too.

5. Hydrogen — Run Python code in Jupyter environment

Want to run Python also in your Atom Editor with any Jupyter Kernel? There is a way for that too. We just need to install “Hydrogen” using the same method as before. Once Hydrogen is installed you can use it by:

  • Run the command on which your cursor is on using Ctrl+Enter.
  • Select any Kernel from the Kernel Selection Screen. I select pyt kernel from the list.
  • Now I can continue working in pyt kernel.
Image for post
Image for post
Runnin command using Ctrl+Enter will ask you which environment to use.

Sometimes it might happen that you don’t see an environment/kernel in Atom. In such cases, you can install ipykernel to make that kernel visible to Jupyter as well as Atom.

Here is how to make a new kernel and make it visible in Jupyter/Atom:

conda create -n exampleenv python=3.7
conda activate exampleenv
conda install -c anaconda ipykernel
python -m ipykernel install --user --name=exampleenv

Once you run these commands, your kernel will be installed. You can now update the Atom’s kernel list by using:

Ctrl+Shift+P >Hydrogen: Update Kernels

Image for post
Image for post

And your kernel should now be available in your Atom editor.

6. Search Stack Overflow from your Text Editor

Stack Overflow is an integral part of any developer’s life. But you know what the hassle is? To leave the coding environment and go to Chrome to search for every simple thing you need to do. And we end up doing it back and forth throughout the day. So, what if we can access Stack Overflow from Atom? You can do precisely that through the ask-stack package, which lets one search for questions on SO. We can access it using Ctrl+Alt+A

Image for post
Image for post
Access Stack Overflow in Atom using Ctrl+Alt+A.

Some other honorable mentions of packages you could use are:

  • Teletype: Do Pair Coding.
  • Linter: Checks code for Stylistic and Programmatic errors. To enable linting in Python, You can use “linter” and “python-linters”.
  • Highlight Selected: Highlight all occurrences of a text by double-clicking or selecting the text with a cursor.
  • Atom-File-Icons: Provides you with file icons in the left side tree view. Looks much better than before, right?
Image for post
Image for post
Icons for files

Conclusion

In this post, I talked about how I use Atom in my Python Development flow.

There are a plethora of other packages in Atom which you may like, and you can look at them to make your environment even more customizable. Or one can even write their own packages as well as Atom is called as the “Most Hackable Editor”.

If you want to learn about Python and not exactly a Python editor, I would like to call out an excellent course on Learn Intermediate level Python from the University of Michigan. Do check it out. Also, here are my course recommendations to become a Data Scientist in 2020.

I am going to be writing more beginner-friendly posts in the future too. Follow me up at Medium or Subscribe to my blog to be informed about them. As always, I welcome feedback and constructive criticism and can be reached on Twitter @mlwhiz.

Also, a small disclaimer — There might be some affiliate links in this post to relevant resources, as sharing knowledge is never a bad idea.

Sign up for The Daily Pick

By Towards Data Science

Hands-on real-world examples, research, tutorials, and cutting-edge techniques delivered Monday to Thursday. Make learning your daily ritual. Take a look

By signing up, you will create a Medium account if you don’t already have one. Review our Privacy Policy for more information about our privacy practices.

A Medium publication sharing concepts, ideas, and codes.

Image for post
Image for post
World dynamics, as defined in the World2 model of Forrester (1971).

The first doomsday computer model, based on Forrester’s System Dynamics, revisited with the R programming language

“In 1972 a small paperbound book called ‘The Limits to Growth’ was published with much fanfare […] the book warned that the world was steering a course for disaster. Without a drastic change in direction, the human population would run out of food and natural resources, or else would choke on its own pollution, within 50 or 100 years” (Hayes, 1993).

Jay Wright Forrester (1918–2016) was an American computer engineer and systems scientist, credited as the “founding father” of system dynamics. Although less known by the general public than the Meadows couple who wrote ‘The Limits to Growth’ (Meadows et al., 1972), Forrester’s world dynamics model World2 (Forrester, 1971) was the basis of all subsequent models predicting a collapse of our socio-technological-natural system by the mid 21st century. This article revisits this seminal work from a programming perspective, by rewriting the outdated DYNAMO code into R. …


It’s Not What You Think

Image for post
Image for post
Photo by Nikolas Noonan on Unsplash

Thomas Nagel wrote about what it must be like to be a bat. His paper is a welcome reflection for data scientists living in a pandemic world. He presents us with the hypothesis below.

“I assume we all believe that bats have experience. After all, they are mammals, and there is no more doubt that they have experience than that mice or pigeons or whales have experience.”

He continues, “Bats, although more closely related to us than those other species, nevertheless present a range of activity and a sensory apparatus so different from ours that the problem I want to pose is exceptionally vivid (though it certainly could be raised with other species). …


Image for post
Image for post
An Apache Airflow Pipeline. Souce: Unsplash

Recap

In the first post of our series, we learned a bit about Apache Airflow and how it can help us build not only Data Engineering & ETL pipelines, but also other types of relevant workflows within advanced analytics, such as MLOps workloads.

We skimmed briefly through some of its building blocks, namely Sensors, Operators, Hooks and Executors. These components provide the basic foundation for working with Apache Airflow. Back then, we worked with the SequentialExecutor, the simplest possible Airflow setup. Having support for running only one task at a time, it is used mainly for simple demonstrations. …


And they have nothing to do with your technical skills

Image for post
Image for post
Image by Arek Socha on Pixabay

Finally, all data were cleansed and ready to analyze. Andy started overenthusiastically to visualize the data to get a first impression of the data. He had many dimensions and variables such that he spent several days visually analyzing them and determining the best methods to apply. At the end of that week, the team manager told him that he would need a draft presentation about the outcomes next Tuesday because the team manager had to present it in one week to a steering committee.

Andy told him that he has no results yet. But there was no space for negotiations. On Tuesday, conclusions had to be delivered and integrated into a PowerPoint presentation. …


What to consider to make your data warehouse and data lake future-proof & how the separation of storage and compute was approached by Snowflake, Amazon, Google, SAP, and IBM

Image for post
Image for post
Photo by John Schnobrich on Unsplash

Not so long ago, establishing an enterprise data warehouse involved a project that would take months or even years. These days, with cloud computing, you can easily register for a SaaS or PaaS offering provided by one of the cloud vendors, and shortly after you can start building your schemas and tables. In this article, I will discuss the key features to consider when migrating a data warehouse to the cloud and why is it a smart choice to pick one that separates storage from compute.

What does it mean to separate storage and compute?

From a single server to a data warehouse cluster

It all boils down to the difference between scale-out & scale-in vs. scale-up & scale-down. In older database and data warehouse solutions the storage and compute reside within a single (often large & powerful) server instance. This may work well until this single server instance would reach its maximum compute or storage capacity. In such cases, in order to accommodate the increased workloads, you could scale-up, i.e. exchange the CPU, RAM, or storage disks to ones with a larger capacity — with cloud services it would mean switching to a larger instance. Analogically, if your single instance is too large, to save money, you could exchange it for a smaller one, i.e. scale-down. …